A software release life cycle may involve many stages of development for an application or other piece of computer software. For example, the stages may include an initial development stage, a generally available release of a first version, and a release of updated versions (including bug fixes and/or modified features) for the remainder of the life cycle. Throughout the various stages, the software may undergo software testing to determine whether it works as expected. In the initial development stage, an alpha release of the software may be tested internally by the developer. Based on the testing of the alpha release, the software may be modified to yield greater stability and other improvements. Eventually, a beta release that incorporates these improvements and typically includes a complete set of features may be released to a set of users for additional testing. Depending on whether the beta is a closed beta or open beta, the set of users may be highly restricted (e.g., based on invitation by the developer) or instead open to a larger group. The users may test the beta release for usability, stability, performance, or any other desired characteristics, and the users may report their findings (e.g., any encountered bugs or usability issues) to the developer. The developer may improve the software based on the findings from the users. Multiple versions of the beta release may be issued and retested as improvements are made by the developer.
Prior to releasing a generally available version of an application, the developer may desire to test the application on many different configurations of computing devices on which the application may potentially be executed by users. In some cases, the developer may test the application using a physical device farm that includes numerous different types of devices. However, for some target platforms, the number of potential devices or device configurations is too large for the developer to perform thorough in-house testing. Additionally, new devices that implement the target platform may become available without the knowledge of the developer.
While embodiments are described herein by way of example for several embodiments and illustrative drawings, those skilled in the art will recognize that embodiments are not limited to the embodiments or drawings described. It should be understood, that the drawings and detailed description thereto are not intended to limit embodiments to the particular form disclosed, but on the contrary, the intention is to cover all modifications, equivalents and alternatives falling within the spirit and scope as defined by the appended claims. The headings used herein are for organizational purposes only and are not meant to be used to limit the scope of the description or the claims. As used throughout this application, the word “may” is used in a permissive sense (i.e., meaning “having the potential to”), rather than the mandatory sense (i.e., meaning “must”). Similarly, the words “include,” “including,” and “includes” mean “including, but not limited to.”